Integration Guide KUKA (KRC)

Note: We strongly recommend that you read the Robot communication overview before this integration guide.

Note 2: This guide describes Robot module 1.12.0 for Bin Picking Studio (BPS) 1.12. We strongly recommend that you use the latest version of the Robot module. You can download the Robot module from the official Photoneo website. The Robot module support matrix lists the supported versions of all Robot modules.

Contents

• 1 Prerequisites

• 2 Robot controller setup

  • 2.1 Controller configuration

    • 2.1.1 Network configuration

  • 2.2 Robot module installation

    • 2.2.1 Ethernet KRL configuration

    • 2.2.2 Loading the Robot module files

    • 2.2.3 Robot State Server configuration

    • 2.2.4 Tool and base selection

• 3 Robot module

  • 3.1 Robotic API

    • 3.1.1 Connection procedures

    • 3.1.2 Communication procedures

    • 3.1.3 Bin picking procedures

    • 3.1.4 Variables, flags and other resources used by the module

    • 3.1.5 Info data

  • 3.2 Example programs

    • 3.2.1 Basic bin picking example

    • 3.2.2 Multiple Vision Systems example

    • 3.2.3 Change solution example

    • 3.2.4 Calibration example

  • 3.3 Error handling

  • 3.4 Robot State Server

  • 3.5 Version and brand ID

• 4 Running the basic bin picking example program

  • 4.1 Prerequisites

  • 4.2 Bin picking routine execution settings

  • 4.3 Reteach the robot poses

  • 4.4 Runtime

    • 4.4.1 Monitoring bin picking related variables

• 5 Migration guide

  • 5.1 Robot module update

1 Prerequisites

• KUKA KR C4 robot controller with KSS 8.3 or higher, or a KUKA KR C5 robot controller

• KUKA.EthernetKRL option - at least v2.2.8 is required

• Bin Picking Studio 1.12

The KUKA Option Package (.kop) shipped with the Robot module can be installed on KSS 8.5 and KSS 8.6. On other system versions, copy the individual files (see 2.2.2).

To check whether EthernetKRL is installed, switch to Expert mode and open Menu -> Start-up -> Additional software:

If EthernetKRL is not installed on your robot controller, contact your local KUKA sales or support representative.

2 Robot controller setup

2.1 Controller configuration

Use the KUKA Line Interface (KLI) of the robot controller to connect to the vision controller. Connect the KLI port either directly to the Robot port of the vision controller or through a network switch.

2.1.1 Network configuration

The Robot module communicates with BPS over TCP/IP. It uses two connections:

Connection	EKI configuration	Port	Direction
Action Request Client	pho_bp_client.xml	54601	The robot controller connects to the Action Request Server on the vision controller and sends requests.
Robot State Server	pho_state_server.xml	54602	The vision controller connects to the robot controller and receives the robot state.

Set the IP address of the robot controller: switch to Expert mode and open Menu -> Start-up -> Network configuration:

Select a fixed IP address. Enter the IP address and subnet mask of the robot controller. In this guide, the robot controller uses 192.168.1.2 / 255.255.255.0 and the vision controller uses 192.168.1.1 / 255.255.255.0:

Save the changes. Reboot the control PC to apply them: open Menu -> Shutdown -> Reboot control PC:

Vision controller side: By default, the Action Request Server of BPS listens on port 11003, but the Robot module connects to port 54601. On the Network page of BPS, enter 54601 in the Action Request Server port (ARS port) field. Leave the Robot State Server port (RSS port) field empty. The Studio then connects to port 54602 by default. For details, see Network setup. To check that the robot controller is reachable, use the Test connection button on the Network page.

2.2 Robot module installation

The Robot module is shipped in two forms:

• KUKA Option Package/Photoneo_BPS_1_12_0.kop - installs all files at once.

• KUKA BPS Module Individual Files/ - the same files, for manual copying.

Shipped folder / file	Location on the robot controller	Note
EthernetKRL/pho_bp_client.xml, EthernetKRL/pho_state_server.xml	C:\KRC\Roboter\Config\User\Common\EthernetKRL\	EKI configuration. Edit the IP addresses (see 2.2.1).
Photoneo/pho_common.src, Photoneo/pho_common.dat	KRC:\R1\PHOTONEO	Communication and API functions. Do not edit.
Photoneo/pho_motion.src, Photoneo/pho_motion.dat	KRC:\R1\PHOTONEO	Execution of the bin picking trajectories, speed settings. Do not edit the .src.
Photoneo/pho_state_server.src, Photoneo/pho_state_server.dat	KRC:\R1\PHOTONEO	Robot State Server. Do not edit.
Photoneo/customer_definitions.src	KRC:\R1\PHOTONEO	Gripper procedures and bin picking settings. Edit this file.
Program/*.src, Program/*.dat	KRC:\R1\Program	Example programs, teach.src and CollDetect_UserAction.src (see 3.2).

Robot module files in R1/PHOTONEO (left) and the example programs in R1/Program (right)

2.2.1 Ethernet KRL configuration

Both XML files must be in C:\KRC\Roboter\Config\User\Common\EthernetKRL\:

In pho_bp_client.xml, enter the IP address of the vision controller into the <IP> tag of the <EXTERNAL> section. Keep the port 54601:

In pho_state_server.xml, enter the IP address of the robot controller into the <IP> tag of the <INTERNAL> section. Keep the port 54602:

Do not change the other entries. These files configure the flags that the Robot module uses (see 3.1.4). The Robot State Server runs in the Submit interpreter (<ENVIRONMENT>Submit</ENVIRONMENT>).

Save the changes and reboot the control PC to apply the new EthernetKRL configuration:

2.2.2 Loading the Robot module files

There are three ways to copy the Robot module to the robot controller.

KUKA Option Package (recommended for a first installation)

1. Switch to Expert mode and copy Photoneo_BPS_1_12_0.kop from a USB drive to D:\KUKA_OPT.

2. Open Menu -> Start-up -> Additional software and press New software.

3. Select the Photoneo package in the list and press Install. Confirm the installation and the following project changes with Yes.

Installation of the KUKA Option Package

The package copies the PHOTONEO folder to R1, the example programs to R1/Program and the XML files to the EthernetKRL folder. Edit the IP addresses in the XML files afterwards (see 2.2.1).

Individual files from a USB drive

Use this method to update only some files (for example pho_common.src / pho_common.dat) without changing your own programs. In the Navigator, use Edit -> Copy / Paste to copy the files to the locations listed in 2.2. You can ignore compilation errors until all files are copied.

WorkVisual

In WorkVisual, connect to the robot controller (Programming and diagnosis workspace). Copy the files into the KRC\R1\PHOTONEO and KRC\R1\Program folders of the project and use Transfer changes. You must still copy the XML files to the EthernetKRL folder on the teach pendant, or set them in the Configuration and commissioning workspace.

Transfer of the Robot module files with WorkVisual

We recommend that you hide the PHOTONEO folder from the operator. In Expert mode, set the System flag in the properties of the folder:

2.2.3 Robot State Server configuration

The Robot State Server runs in the Submit interpreter. It is needed for calibration, for hand-eye scanning and to show the robot in the Studio.

Switch to Expert mode and deselect the Submit interpreter before editing:

Open R1/System/sps.sub and add two lines: PHO_StatePublisherInit() to the USER INIT fold and PHO_PublishState() to the USER PLC fold:

Select and start the Submit interpreter again. The Robot State Server then waits for the vision controller to connect. For the data that it sends, see 3.4 Robot State Server.

2.2.4 Tool and base selection

The Robot State Server sends the tool pose only when a TOOL and a BASE are selected. If they are not selected (T? and B?), it sends only the joint positions and shows the message “Set TOOL and BASE frames to get PHO STATE SERVER running properly!”:

Select $NULLFRAME for both TOOL and BASE. The reported tool pose is then the pose of the bare flange in the robot base frame. Calibration and hand-eye scanning expect this pose:

For picking with the Trajectory request, the selected tool does not matter. The trajectory is sent as joint positions, and the Studio handles the tool offset. For picking based on the Object pose request, the robot program must use a tool that matches the real gripper.

3 Robot module

The Robot module is a set of KRL functions. You can call them from any KRL program. pho_common, pho_motion and pho_state_server are the communication part. Do not edit them. You can adapt customer_definitions.src and the example programs to your application.

3.1 Robotic API

Note: We strongly recommend that you read Photoneo robotic API and Action requests before this section.

3.1.1 Connection procedures

Procedure

Description

PHO_ConnectToVc()

Opens the EKI channel pho_bp_client and waits until the connection to the Action Request Server is established ($FLAG[101] = TRUE). Then it sends the brand ID, resets the request tracking and shows the message “Connected to Bin Picking Studio”. If $FLAG[101] is already TRUE, the procedure returns immediately. The IP address and port come from pho_bp_client.xml, so you do not need to set them in the program. Call it at the start of the program and whenever $FLAG[101] is FALSE (see the example programs).

3.1.2 Communication procedures

All procedures are defined in pho_common.src. Most requests take the vision system ID from the global variable vision_system_id. Set it before you call the request. Procedures declared as INT return the error code. All procedures also write it to the global variable error_code. For the meaning of each request, see Action requests.

Bin picking requests

Procedure	ID	Inputs	Outputs
PHO_RequestInit(start_pose, end_pose) Initialization	4	vision_system_id start_pose, end_pose - AXIS in degrees	error_code
PHO_RequestScan() Scan	1	vision_system_id	None - only sends the request. Receive the result with PHO_WaitForScan().
INT PHO_WaitForScan() Wait for scan completion		-	Return value, error_code
PHO_RequestTrajectory() Trajectory	2	vision_system_id	None - only sends the request. Receive the result with PHO_ReceiveTrajectory().
INT PHO_ReceiveTrajectory() Receive trajectory		-	Return value, error_code, trajectory stored for PHO_PickPart(), Info data keys 1-7 (see 3.1.5)
INT PHO_RequestPickFailed() Pick-failed	7	vision_system_id	Return value, error_code
INT PHO_RequestObjectPose() Object pose	8	vision_system_id	Return value, error_code, object_pose (POS, X/Y/Z in mm, A/B/C in degrees, robot base frame), Info data keys 4-7
INT PHO_RequestEnvChange(env_id) Change scene state	15	env_id - scene state ID	Return value, error_code
INT PHO_RequestGetStatus() Get vision system status	21	vision_system_id	Return value, error_code, num_of_localized_objects, num_of_ready_objects, vs_still_processing
PHO_RequestCapture() Trigger scan	28	vision_system_id, current tool pose ($POS_ACT_MES, a TOOL and BASE must be selected)	error_code. The procedure waits for the response itself.
INT PHO_ReuseScan() Localize on the last scan	29	vision_system_id	Return value, error_code. The procedure waits for the response itself - continue with PHO_RequestTrajectory().
INT PHO_RequestChangeBBox(bbox_id) Change bounding box	32	vision_system_id, bbox_id	Return value, error_code
PHO_RequestBinLocator()	3	vision_system_id	error_code. For Photoneo internal use only.

Calibration requests

Procedure	ID	Inputs	Outputs
INT PHO_RequestCalibAdd() Add calibration point	5	-	Return value, error_code
INT PHO_RequestCalibStart(arg_solution_id, arg_vision_system_id) Start automatic calibration	25	arg_solution_id, arg_vision_system_id	Return value, error_code
INT PHO_RequestCalibSave() Save automatic calibration result	27	-	Return value, error_code, calib_error_um (µm), calib_pose (POS, camera pose, mm / degrees)
INT PHO_RequestCalibStop() Stop automatic calibration	26	-	Return value, error_code

Solution requests

Procedure	ID	Inputs	Outputs
PHO_RequestChangeSol(solution_id) Change solution	9	solution_id	error_code
INT PHO_RequestStartSol(solution_id) Start solution	10	solution_id	Return value, error_code
INT PHO_RequestStopSol() Stop solution	11	-	Return value, error_code
INT PHO_RequestGetRunningSol(solution_id) Get running solution	12	-	Return value, error_code, solution_id (OUT parameter, -1 on error), running_sol_id

Other requests

Procedure	ID	Inputs	Outputs
INT PHO_RequestCommCheck() Communication check	100	-	Return value, error_code

Start and end pose of the Initialization request

PHO_RequestInit needs the start and end pose as AXIS (joint positions in degrees). If the poses are taught as E6POS (for example in teach.src), convert them with INVERSE(), as in the example programs:

temp = {A1 0, A2 0, A3 0, A4 0, A5 0, A6 0}
conversion_status = 0
IF FSTART1.TOOL_NO == 0 THEN
  $TOOL = $NULLFRAME
ELSE
  $TOOL = TOOL_DATA[FSTART1.TOOL_NO]
ENDIF
IF FSTART1.BASE_NO == 0 THEN
  $BASE = $NULLFRAME
ELSE
  $BASE = BASE_DATA[FSTART1.BASE_NO]
ENDIF
start_joint_pos = INVERSE(XSTART1, temp, conversion_status)

Order of requests: The Robot module allows only one request at a time. Receive the response to every request before you send the next request. For the Scan and Trajectory requests, this means PHO_WaitForScan() / PHO_ReceiveTrajectory(). If the order is wrong, the module stops the program (see 3.3).

Debug messages: Set the global variable PHO_DEBUG to TRUE to show a message for every sent request and received response.

3.1.3 Bin picking procedures

Procedures in pho_common.src / pho_motion.src (do not edit)

Procedure	Description
PHO_PickPart()	Executes the bin picking trajectory received by the last successful PHO_ReceiveTrajectory(). It runs the received operations in order: trajectory segments (PHO_ExecuteTrajectory) and gripper commands (PHO_ExecuteGripperCmd). The robot must be in the start pose when the procedure is called. At the end, the robot is in the end pose with the object in the gripper. A trajectory can be executed only once. If you call PHO_PickPart() without a new successful trajectory, the program stops. Warning: When several vision systems use different start poses, make sure the robot is in the start pose of the vision system the trajectory was requested for.
PHO_ExecuteTrajectory(approximation_distance, traj_number, traj_size, vel, acc)	Executes one trajectory segment as a PTP_SPLINE of SPTP points (up to 100 points). Segments of type CNT use the approximation distance approx_dist (mm). Segments of type FINE use 0 and stop at the last point. vel and acc are taken from the velocities[] / accelerations[] arrays (% of maximum). Called by PHO_PickPart().
PHO_ExecuteGripperCmd(gripper_cmd)	Calls the gripper procedure that matches the command received from the Studio: 1 = Attach, 2 = Detach, 3 = User 1, 4 = User 2, 5 = User 3. Called by PHO_PickPart().

Procedures in customer_definitions.src (edit to fit your application)

Procedure	Description
PHO_BinpickingSettings()	Sets approx_dist (default 75 mm) and the speed (velocities[], default 100 %) and acceleration (accelerations[], default 40 %) of each trajectory segment. Call it at the start of the program. See 4.2.
PHO_GripperAttach()	Executed when a waypoint of the grasping method has the Attach gripper action - typically at the Grasp waypoint.
PHO_GripperDetach()	Executed when a waypoint of the grasping method has the Detach gripper action. Usually, the robot program calls it during placing instead.
PHO_GripperUser_1(), PHO_GripperUser_2(), PHO_GripperUser_3()	Executed when a waypoint of the grasping method has the User 1, User 2 or User 3 gripper action.

The gripper procedures are empty by default. Add your I/O commands, for example $OUT[101] = TRUE.

3.1.4 Variables, flags and other resources used by the module

Flags - set by EthernetKRL and read-only for the robot program (except where noted). Do not use them for other purposes.

Flag	Set by	Meaning
$FLAG[100]	pho_state_server.xml (ALIVE)	TRUE while the vision controller is connected to the Robot State Server.
$FLAG[101]	pho_bp_client.xml (ALIVE)	TRUE while the robot controller is connected to the Action Request Server. The example programs reconnect when it is FALSE.
$FLAG[102]	pho_bp_client.xml (receive)	Response data from the Action Request Server is waiting in the receive buffer. Reset by the Robot module.
$FLAG[103]	pho_state_server.xml (receive)	Data received on the Robot State Server connection (not used).

Other resources

• Interrupts 100 and 101 - declared in the Submit interpreter by PHO_StatePublisherInit().

• Timer $TIMER[64] - used by PHO_PublishState() for the 100 ms period.

• EKI channels pho_bp_client and pho_state_server.

Global variables (declared in pho_common.dat and pho_motion.dat)

Variable	Type	Meaning
vision_system_id	INT	Vision system ID used by the requests. Set it before the request.
error_code	INT	Error code of the last request (see 3.3).
object_pose	POS	Object pose from PHO_RequestObjectPose() (mm, degrees, robot base frame).
calib_pose	POS	Calibration result from PHO_RequestCalibSave() (mm, degrees).
PHO_DEBUG	BOOL	TRUE = show a message for every request and response. Default FALSE.
info_data[10]	INT	Values of Info data keys that the module does not know (for future use).
approx_dist	REAL	Approximation distance of CNT trajectory segments in mm (pho_motion.dat).
velocities[10], accelerations[10]	REAL	Speed and acceleration of trajectory segments 1-10 in % (pho_motion.dat).

The Info data variables are listed in 3.1.5. Other variables in pho_common.dat (trajectory_buffer, trajectory_sizes, gripper_buffer, binpicking_operations, num_of_oper) are used internally. Do not write to them.

3.1.5 Info data

The module stores the values returned by the requests in global variables in pho_common.dat. To see which request returns which key, see Info data. A variable keeps its value until a response contains the key again.

Key	Info data	Variable	Type	Unit in robot
1	Tool point invariance	tool_point_inv	INT	index
2	Gripping point ID	gripping_point_id	INT	index
3	Gripping point invariance	gripping_point_inv	INT	index
4	Dimension X	dimension_x	INT	mm
5	Dimension Y	dimension_y	INT	mm
6	Rotation Z	dimension_angle	REAL	degrees (converted from mrad)
7	Neural network label	nn_label	INT	integer
8	Max Z height	max_z_height	REAL	mm (not sent by the BPS)
9	Tilt	tilt	REAL	not sent by the BPS
10	Calibration accuracy	calib_error_um	REAL	µm
11	Running solution ID	running_sol_id	INT	integer
12	Number of localized objects	num_of_localized_objects	INT	integer
13	Number of ready objects	num_of_ready_objects	INT	integer
14	Processing state	vs_still_processing	INT	0 / 1

3.2 Example programs

The example programs are located in R1/Program. Each program uses the API in the correct order and contains basic error handling. Adapt the error handling to your application.

Program	Purpose
basic_application.src	Basic bin picking: connect, initialize one vision system, scan, request trajectory, pick, place. See 3.2.1.
multi_vision_systems.src	Bin picking with two vision systems, switched in every cycle with vision_system_id.
change_solution.src	Bin picking that switches between two solutions after every pick with PHO_RequestChangeSol().
change_environment.src	Bin picking that switches between two scene states after every pick with PHO_RequestEnvChange().
change_bbox.src	Bin picking that switches between two bounding boxes after every pick with PHO_RequestChangeBBox().
get_status.src	Basic bin picking that calls PHO_RequestGetStatus() several times before requesting the trajectory.
reuse_scan.src	Bin picking with two vision systems on the same scene: vision system 1 scans, vision system 2 localizes on the same scan with PHO_ReuseScan().
hand_eye.src	Bin picking with a robot-mounted (hand-eye) sensor: the scan is triggered when the robot reaches the scanning pose.
meshing.src	Hand-eye meshing: several PHO_RequestCapture() calls while the robot moves over the scene, then PHO_RequestScan() starts the localization. Requires a MotionCam-3D.
calibration.src	Calibration with 9 taught poses (PHO_RequestCalibAdd()). Automatic calibration is switched off by default. See 3.2.4.
calibration_he.src	The same calibration for a hand-eye vision system, with automatic calibration switched on.
comm_check.src	Repeated PHO_RequestCommCheck() calls with automatic reconnection.
teach.src	Holds the start and end poses START1, END1, START2, END2 used by the other examples. Not meant to be run.
CollDetect_UserAction.src	Empty KUKA collision detection user action program. It is shipped because the example programs use collision detection parameters. Not used by the Robot module.

Important: The feature used by an example must be set up in the Studio first - for example change_bbox.src needs at least two bounding boxes in the solution.

Note: When a PLC is the high-level controller, you can split the code of the main program into separate programs and call them from cell.src.

3.2.1 Basic bin picking example

The program connects to the vision controller, initializes vision system 1 and triggers the first scan. Then it runs in a loop: wait for the scan, request and receive the trajectory, move to the start pose, pick, trigger the next scan while moving to the placing pose, and place. After errors that need a new initialization (203, 1, 4), it initializes and scans again. After errors 201 and 202, it only scans again.

File: basic_application.src

DEF basic_application( )
  ;FOLD INI;%{PE}

    BOOL trajectory_ok, scan_ok
    E6AXIS temp
    AXIS start_joint_pos, end_joint_pos
    INT conversion_status, scan_status, trajectory_status

    ;FOLD BASISTECH INI
      GLOBAL INTERRUPT DECL 3 WHEN $STOPMESS == TRUE DO IR_STOPM ( )
      INTERRUPT ON 3
      BAS (#INITMOV, 0 )
    ;ENDFOLD (BASISTECH INI)
    ;FOLD USER INI
      ;Make your modifications here

    ;ENDFOLD (USER INI)
  ;ENDFOLD (INI)

;FOLD PTP HOME Vel=100 % DEFAULT;%{PE}
;FOLD Parameters ;%{h}
;Params IlfProvider=kukaroboter.basistech.inlineforms.movement.old; Kuka.IsGlobalPoint=False; Kuka.PointName=HOME; Kuka.BlendingEnabled=False; Kuka.MoveDataPtpName=DEFAULT; Kuka.VelocityPtp=100; Kuka.CurrentCDSetIndex=0; Kuka.MovementParameterFieldEnabled=True; IlfCommand=PTP
;ENDFOLD
$BWDSTART = FALSE
PDAT_ACT = PDEFAULT
FDAT_ACT = FHOME
BAS(#PTP_PARAMS, 100.0)
SET_CD_PARAMS (0)
PTP XHOME
;ENDFOLD

LOOP

    ; Turn TRUE/FALSE Photoneo Debug Messages
    PHO_DEBUG = FALSE

    ; If robot controller has not been connected to vision controller, connect, initialize and trigger first scan
    CONTINUE
    IF NOT $FLAG[101] THEN

        ; Ensure that HOME position is reachable from the last placing point (P3 in this case) without collision since transition from P3 to HOME happens when binpicking error occurs
      ;FOLD PTP HOME  Vel= 100 % DEFAULT;%{PE}%MKUKATPBASIS,%CMOVE,%VPTP,%P 1:PTP, 2:HOME, 3:, 5:100, 7:DEFAULT
        $BWDSTART = FALSE
        PDAT_ACT = PDEFAULT
        FDAT_ACT = FHOME
        BAS (#PTP_PARAMS, 100 )
        $H_POS = XHOME
        PTP XHOME
      ;ENDFOLD

        ; Apply Binpicking settings
        PHO_BinpickingSettings()

        ; Set Vision System ID (default = 1)
        vision_system_id = 1

        ; Connect to Vision Controller
        PHO_ConnectToVc()

      ;FOLD Convert Start Pose to AXIS representation
        temp = {A1 0, A2 0, A3 0, A4 0, A5 0, A6 0}
        conversion_status = 0
        IF FSTART1.TOOL_NO == 0 THEN
          $TOOL = $NULLFRAME
        ELSE
          $TOOL = TOOL_DATA[FSTART1.TOOL_NO]
        ENDIF
        IF FSTART1.BASE_NO == 0 THEN
          $BASE = $NULLFRAME
        ELSE
          $BASE = BASE_DATA[FSTART1.BASE_NO]
        ENDIF
        start_joint_pos = INVERSE(XSTART1, temp, conversion_status)
      ;ENDFOLD

      ;FOLD Convert End Pose to AXIS representation
        temp = {A1 0, A2 0, A3 0, A4 0, A5 0, A6 0}
        conversion_status = 0
        IF FEND1.TOOL_NO == 0 THEN
          $TOOL = $NULLFRAME
        ELSE
          $TOOL = TOOL_DATA[FEND1.TOOL_NO]
        ENDIF
        IF FEND1.BASE_NO == 0 THEN
          $BASE = $NULLFRAME
        ELSE
          $BASE = BASE_DATA[FEND1.BASE_NO]
        ENDIF
        end_joint_pos = INVERSE(XEND1, temp, conversion_status)
      ;ENDFOLD

        ; Send Initialization Request
        PHO_RequestInit(start_joint_pos, end_joint_pos)

        ; Trigger first scan
        PHO_RequestScan()

        ; Initial Wait
        WAIT SEC 5

    ENDIF

    ; Reset trajectory_ok flag
    trajectory_ok = FALSE

    ; Scanning & Planning loop
    WHILE trajectory_ok == FALSE

        ; Wait for first scan
        scan_status = PHO_WaitForScan()

        ; If Scan OK, request trajectory
        IF (scan_status == PHO_OK) THEN

            PHO_RequestTrajectory()
            trajectory_status = PHO_ReceiveTrajectory()

            SWITCH trajectory_status
                CASE PHO_NOT_INITIALIZED

                    ; Send Initialization Request
                    PHO_RequestInit(start_joint_pos, end_joint_pos)

                    ; Send Scan Request
                    PHO_RequestScan()

                    ; Initial Wait
                    WAIT SEC 5

                CASE PHO_SERVICE_ERR

                    ; Send Initialization Request
                    PHO_RequestInit(start_joint_pos, end_joint_pos)

                    ; Send Scan Request
                    PHO_RequestScan()

                    ; Initial Wait
                    WAIT SEC 5

                CASE PHO_BAD_DATA

                    ; Send Initialization Request
                    PHO_RequestInit(start_joint_pos, end_joint_pos)

                    ; Send Scan Request
                    PHO_RequestScan()

                    ; Initial Wait
                    WAIT SEC 5

                CASE PHO_PLANNING_FAILED

                    ; Send Scan Request
                    PHO_RequestScan()

                CASE PHO_NO_PART_FOUND

                    ; Send Scan Request
                    PHO_RequestScan()

                CASE PHO_OK

                    ; Set blocking flag to true to exit loop
                    trajectory_ok = TRUE

                DEFAULT
                    msgQuit("Trajectory Error: %1 - Check Photoneo Robotic API Documentation for more info", "BIN PICKING CLIENT", trajectory_status)

            ENDSWITCH

        ; If scan failed, reinitialize and scan again
        ELSE

            ; Send Initialization Request
            PHO_RequestInit(start_joint_pos, end_joint_pos)

            ; Send Scan Request
            PHO_RequestScan()

            ; Initial Wait
            WAIT SEC 5

        ENDIF

    ENDWHILE


    ; Move to Start Position
;FOLD SPTP START1 CONT Vel=100 % PDAT4 Tool[1]:Tool1 Base[0] ;%{PE}
;FOLD Parameters ;%{h}
;Params IlfProvider=kukaroboter.basistech.inlineforms.movement.spline; Kuka.IsGlobalPoint=False; Kuka.PointName=START1; Kuka.BlendingEnabled=True; Kuka.MoveDataPtpName=PDAT4; Kuka.VelocityPtp=100; Kuka.VelocityFieldEnabled=True; Kuka.ColDetectFieldEnabled=True; Kuka.CurrentCDSetIndex=0; Kuka.MovementParameterFieldEnabled=True; IlfCommand=SPTP
;ENDFOLD
SPTP XSTART1 WITH $VEL_AXIS[1] = SVEL_JOINT(100.0), $TOOL = STOOL2(FSTART1), $BASE = SBASE(FSTART1.BASE_NO), $IPO_MODE = SIPO_MODE(FSTART1.IPO_FRAME), $LOAD = SLOAD(FSTART1.TOOL_NO), $ACC_AXIS[1] = SACC_JOINT(PPDAT4), $APO = SAPO_PTP(PPDAT4), $GEAR_JERK[1] = SGEAR_JERK(PPDAT4), $COLLMON_TOL_PRO[1] = USE_CM_PRO_VALUES(0) C_Spl
;ENDFOLD

    ; Pick part
    PHO_PickPart ( )

    ; Send Next Scan Request when P1 pose is reached
    TRIGGER WHEN DISTANCE=1 DELAY=0 DO PHO_RequestScan() PRIO=100

    ; Placing
;FOLD SPTP P1 CONT Vel=100 % PDAT1 Tool[1]:Tool1 Base[0] ;%{PE}
;FOLD Parameters ;%{h}
;Params IlfProvider=kukaroboter.basistech.inlineforms.movement.spline; Kuka.IsGlobalPoint=False; Kuka.PointName=P1; Kuka.BlendingEnabled=True; Kuka.MoveDataPtpName=PDAT1; Kuka.VelocityPtp=100; Kuka.VelocityFieldEnabled=True; Kuka.ColDetectFieldEnabled=True; Kuka.CurrentCDSetIndex=0; Kuka.MovementParameterFieldEnabled=True; IlfCommand=SPTP
;ENDFOLD
SPTP XP1 WITH $VEL_AXIS[1] = SVEL_JOINT(100.0), $TOOL = STOOL2(FP1), $BASE = SBASE(FP1.BASE_NO), $IPO_MODE = SIPO_MODE(FP1.IPO_FRAME), $LOAD = SLOAD(FP1.TOOL_NO), $ACC_AXIS[1] = SACC_JOINT(PPDAT1), $APO = SAPO_PTP(PPDAT1), $GEAR_JERK[1] = SGEAR_JERK(PPDAT1), $COLLMON_TOL_PRO[1] = USE_CM_PRO_VALUES(0) C_Spl
;ENDFOLD


;FOLD SPTP P2 Vel=100 % PDAT3 Tool[1]:Tool1 Base[0] ;%{PE}
;FOLD Parameters ;%{h}
;Params IlfProvider=kukaroboter.basistech.inlineforms.movement.spline; Kuka.IsGlobalPoint=False; Kuka.PointName=P2; Kuka.BlendingEnabled=False; Kuka.MoveDataPtpName=PDAT3; Kuka.VelocityPtp=100; Kuka.VelocityFieldEnabled=True; Kuka.ColDetectFieldEnabled=True; Kuka.CurrentCDSetIndex=0; Kuka.MovementParameterFieldEnabled=True; IlfCommand=SPTP
;ENDFOLD
SPTP XP2 WITH $VEL_AXIS[1] = SVEL_JOINT(100.0), $TOOL = STOOL2(FP2), $BASE = SBASE(FP2.BASE_NO), $IPO_MODE = SIPO_MODE(FP2.IPO_FRAME), $LOAD = SLOAD(FP2.TOOL_NO), $ACC_AXIS[1] = SACC_JOINT(PPDAT3), $APO = SAPO_PTP(PPDAT3), $GEAR_JERK[1] = SGEAR_JERK(PPDAT3), $COLLMON_TOL_PRO[1] = USE_CM_PRO_VALUES(0)
;ENDFOLD

    PHO_GripperDetach ( )

;FOLD SPTP P3 CONT Vel=100 % PDAT2 Tool[1]:Tool1 Base[0] ;%{PE}
;FOLD Parameters ;%{h}
;Params IlfProvider=kukaroboter.basistech.inlineforms.movement.spline; Kuka.IsGlobalPoint=False; Kuka.PointName=P3; Kuka.BlendingEnabled=True; Kuka.MoveDataPtpName=PDAT2; Kuka.VelocityPtp=100; Kuka.VelocityFieldEnabled=True; Kuka.ColDetectFieldEnabled=True; Kuka.CurrentCDSetIndex=0; Kuka.MovementParameterFieldEnabled=True; IlfCommand=SPTP
;ENDFOLD
SPTP XP3 WITH $VEL_AXIS[1] = SVEL_JOINT(100.0), $TOOL = STOOL2(FP3), $BASE = SBASE(FP3.BASE_NO), $IPO_MODE = SIPO_MODE(FP3.IPO_FRAME), $LOAD = SLOAD(FP3.TOOL_NO), $ACC_AXIS[1] = SACC_JOINT(PPDAT2), $APO = SAPO_PTP(PPDAT2), $GEAR_JERK[1] = SGEAR_JERK(PPDAT2), $COLLMON_TOL_PRO[1] = USE_CM_PRO_VALUES(0) C_Spl
;ENDFOLD


ENDLOOP

END

3.2.2 Multiple Vision Systems example

multi_vision_systems.src extends the basic example. It initializes vision systems 1 and 2 (each with its own start and end pose START1/END1 and START2/END2) and switches vision_system_id after every pick. The next scan is triggered for the other vision system, and the robot moves to the start pose of the vision system the trajectory belongs to.

3.2.3 Change solution example

change_solution.src extends the basic example. After every pick it switches between two solutions (solution_id_1 = 1, solution_id_2 = 2) with PHO_RequestChangeSol(). The examples change_environment.src and change_bbox.src work the same way with PHO_RequestEnvChange() and PHO_RequestChangeBBox(). Set the IDs at the top of the programs to match your solution.

3.2.4 Calibration example

calibration.src moves the robot to 9 taught calibration poses (P1 - P9) and adds a calibration point in each with PHO_RequestCalibAdd(). The calibration ball or marker pattern must be mounted and the robot must use $NULLFRAME as tool and base (see 2.2.4).

The behavior depends on the switch AUTO_CALIBRATION_ENABLED at the top of the program:

• FALSE (default in calibration.src): start the calibration on the Calibration page of the Studio first. The program only adds the points. Check the result and save it in the Studio.

• TRUE (default in calibration_he.src): the program does the whole calibration itself. Automatic calibration must be enabled for the vision system, and the vision system must have been calibrated once before.

  1. PHO_RequestStopSol() - stops the running solution.

  2. PHO_RequestCalibStart(solution_to_calibrate, vision_sys_to_calibrate) - starts the calibration (IDs 1 and 1 in calibration.src, solution 3 in calibration_he.src - set them to your solution).

  3. PHO_RequestCalibAdd() in each of the 9 poses.

  4. PHO_RequestCalibSave() - saves the result. calib_error_um contains the calibration accuracy in µm, calib_pose the result.

  5. PHO_RequestCalibStop() - stops the calibration.

If the accuracy is not good enough, saving fails with error 467 (Automatic calibration errors). As a rule, the calibration error should stay below 3 mm. For automatic recalibration, the calibration object must stay where it was during the first calibration. The calibration ball stays mounted in the same place on the flange or gripper and moves with the robot. The marker pattern stays in the same place in the cell.

Start of the automatic calibration in calibration.src:

; Auto Calibration Enabled Switch TRUE/FALSE
AUTO_CALIBRATION_ENABLED = FALSE

; Connect to VC if robot controller has not been connected to VC yet
CONTINUE
IF NOT $FLAG[101] THEN
    ; Connect to Vision Controller
    PHO_ConnectToVc()
ENDIF

; If Auto Calibration is enabled do calib initialization
IF(AUTO_CALIBRATION_ENABLED == TRUE) THEN

    ; Stop deployed solution if running
    status = PHO_RequestStopSol()

    ; Set Solution and Vision System IDs to calibrate
    solution_to_calibrate = 1
    vision_sys_to_calibrate = 1

    ; Start Calibration
    status = PHO_RequestCalibStart(solution_to_calibrate, vision_sys_to_calibrate)
    IF(status <> 0) THEN
        MsgQuit("Calibration Start request Failed! Error code: %1", "BIN PICKING CLIENT", status)
    ELSE
        MsgNotify("Calibration Start request OK!", "BIN PICKING CLIENT")
    ENDIF

ENDIF

3.3 Error handling

Error codes

Every request writes its result to error_code. INT procedures also return it. If the result is not 0, the module shows a notification message with the error name (for example “Error: NO PART FOUND”). Check the error code after every synchronous request and after every procedure that receives a response. All error codes are described in Error codes. The Robot module defines these constants in pho_common.dat:

Constant	Code	Error
PHO_OK	0	No error
PHO_SERVICE_ERR	1	Service error
PHO_UNKNOWN_REQ	2	Unknown request
PHO_COMM_FAILURE	3	Communication error
PHO_BAD_DATA	4	Bad data
PHO_TIMEOUT	5	Timeout (not used by this module)
PHO_UNKNOWN_ERR	200	Unknown error
PHO_PLANNING_FAILED	201	Path planning failed
PHO_NO_PART_FOUND	202	No object found
PHO_NOT_INITIALIZED	203	Vision system not initialized
PHO_EMPTY_SCENE	218	Empty scene
PHO_SCENE_NOT_AV	220	Scene state not found
PHO_SCENE_ACTIVE	221	Scene state already active
PHO_BBOX_ID_INVALID	222	Invalid bounding box
PHO_WRONG_BP_CONFIG	255	Wrong bin picking configuration
PHO_INVALID_SOL	450	Invalid solution ID
PHO_ANOTHER_SOL	451	Another solution running
PHO_SOL_START_ER	452	Deployment not started
PHO_SOL_NOT_RUN	453	Deployment is not running
PHO_SOL_STOP_ERR	454	Deployment did not stop
PHO_SOL_INCOMPL	455	Solution is not complete
PHO_INVAL_BRAND	457	Invalid robot brand
PHO_SOL_RUNNING	458	Solution already running
PHO_SOL_FORBID	459	Solution access unauthorized
PHO_INVALID_VS … PHO_CALIB_FORBID	460 - 469	Automatic calibration errors: PHO_INVALID_VS (460), PHO_CALIB_RUNN (461), PHO_CALIB_N_RUNN (462), PHO_CALIB_STOP_E (463), PHO_CALIB_STRT_E (464), PHO_CALIB_DISABL (465), PHO_NOT_ENOUGH_P (466), PHO_NOT_ENOUGH_A (467 - calibration accuracy above the threshold), PHO_SCANNER_NA (468), PHO_CALIB_FORBID (469)

Other codes (for example calibration point errors 100-134) are shown as “Error: <code>”. Compare them with the number directly.

Communication errors

• PHO_ConnectToVc() waits until the connection is established. There is no timeout.

• If the connection is lost while the module waits for a response, the procedure returns Communication error (3) and $FLAG[101] becomes FALSE. The example programs then reconnect with PHO_ConnectToVc() and initialize again at the start of the next loop.

• If a request cannot be sent, the module shows the acknowledgement message “Error: Communication Failure - SendRequest Failed, check ethernet connection!”.

• A trajectory point with a wrong checksum or point number sets Bad data (4).

Errors that stop the program

The following situations are programming or configuration errors. The module sets error_code to 200 and shows an acknowledgement message in an endless loop. Fix the cause and restart the program:

• a request is sent while another request is still waiting for its response, or a response is received for a request that was not sent,

• the response type does not match the request,

• the response has more than 30 operations, more than 10 trajectory segments or a segment with more than 100 points (check the grasping method),

• an unknown operation type or gripper command is received,

• PHO_PickPart() is called without a new successful trajectory.

3.4 Robot State Server

The Robot State Server (pho_state_server.src) runs in the Submit interpreter (see 2.2.3). It listens on port 54602 of the robot controller, and the vision controller connects to it.

• PHO_StatePublisherInit() (USER INIT) opens the EKI channel pho_state_server and declares two interrupts. When the vision controller connects ($FLAG[100] TRUE), the brand ID is sent. When it disconnects ($FLAG[100] FALSE), the channel is closed and opened again.

• PHO_PublishState() (USER PLC) sends the robot state every 100 ms ($TIMER[64]) while a client is connected:

  • joint positions A1-A6 from $AXIS_ACT (degrees),

  • tool pose X, Y, Z (mm) and A, B, C (degrees) from $POS_ACT_MES - only when a TOOL and BASE are selected (see 2.2.4).

The joint positions are used to show the robot in the Studio. The tool pose is needed for calibration and hand-eye scanning. When the Robot State Server is connected, the Robot State Server status on the Deployment page of the Studio is CONNECTED.

3.5 Version and brand ID

• Robot module version: 1.12.0 (Option Package Photoneo_BPS_1_12_0.kop)

• Brand ID: KUKA_KRC/1.12.0XXXXedcba

The brand ID is sent after every connection: by PHO_ConnectToVc() to the Action Request Server, and by the Robot State Server to the vision controller. The Studio uses it to select the robot brand and the version of the communication interface.

4 Running the basic bin picking example program

4.1 Prerequisites

Before running the Basic bin picking example, make sure that:

1. a BPS solution with one vision system is configured and ready for deployment,

2. the robot controller is set up as described in 2 Robot controller setup and the Robot State Server is connected,

3. the network settings of the Studio are set (ARS port 54601),

4. the vision system is calibrated,

5. the start and end poses and the poses in the program are taught,

6. the gripper procedures are implemented (optional - without them the robot does not actually pick the object) and a placing routine is added.

4.2 Bin picking routine execution settings

PHO_BinpickingSettings() in customer_definitions.src sets the motion parameters of the bin picking routine:

• approx_dist - approximation distance of CNT segments in mm (default 75),

• velocities[1..10] - speed of each segment in % of the maximum (default 100),

• accelerations[1..10] - acceleration of each segment in % of the maximum (default 40).

The default grasping method has 4 segments (Start - Approach - Grasp - Approach - End), so the procedure sets the first 4 values. The first value applies to the first segment, the second value to the second segment, and so on. If you add segments in the grasping method (up to 10), set the matching number of values. Call the procedure at the start of the program, as in the example programs.

Note: If the motion is jerky, try lower accelerations or a different approximation distance.

4.3 Reteach the robot poses

Teach the home, start, and end poses:

• Home - outside the scanning area.

• Start - the gripper approximately above the center of the bin.

• End - similar to the start pose or slightly shifted towards the placing area. Do not put it too far from the bin. This can increase the planning time or cause planning errors.

Teaching the start and end poses

Switch to Expert mode and select teach.src. It contains the start and end poses that the example programs use:

Select the line with the pose to reteach (START1 in this case):

Press Block selection to move the program pointer to the line. Jog the robot to the new pose and press Touch-up:

Confirm with Yes. Repeat these steps for END1.

The examples with two vision systems use the poses START2 and END2. You can add new poses to teach.src the same way. Add the keyword GLOBAL to their declaration in teach.dat so that other programs can use them.

Teaching the home and placing poses

Teach the home pose and the placing poses P1, P2 and P3 directly in basic_application.src.

Warning: Make sure the home pose is reachable from the last placing pose without collision. The robot moves to the home pose directly when the connection is lost.

4.4 Runtime

1. Deploy your solution in the Studio. The Action Request Client status on the Deployment page is DISCONNECTED until the robot program connects. The Robot State Server should already be CONNECTED.

2. Switch the robot to T1 and reduce the override to 10 % for the first run.

3. Select basic_application.src in R1/Program and start it.

4. The robot moves to the home pose, connects and sends the requests. The Action Request Client status changes to CONNECTED, the sensor scans and the localization starts.

5. When a trajectory is received, the robot moves to the start pose and picks the object. Check that the path is correct. If the robot is too far from the object or pushes too deep, adjust the tool point or gripping points in the Studio.

6. Add your placing routine and slowly increase the speed.

Warning: Be ready to stop the robot immediately in case of any problem.

4.4.1 Monitoring bin picking related variables

Use Display -> Variable to watch variables such as error_code, vision_system_id or the Info data variables:

5 Migration guide

To migrate from BPS 1.11.x to BPS 1.12.x, read the general BPS migration guide. For the KUKA KRC Robot module 1.12, you only need to replace the Robot module files (see 5.1). New in 1.12: the variables nn_label, max_z_height and tilt and the example programs comm_check.src and reuse_scan.src.

5.1 Robot module update

1. Back up customer_definitions.src. It contains your gripper procedures and bin picking settings. Also back up teach.src / teach.dat and your own programs.

2. Replace the files in R1/PHOTONEO with the new ones (except customer_definitions.src), either with the new Option Package or by copying the individual files (see 2.2.2).

3. Copy your changes from the old customer_definitions.src into the new one.

4. Replace pho_bp_client.xml and pho_state_server.xml with the new ones and enter the IP addresses again. Reboot the control PC.

5. Check the API changes listed above and adapt your program if needed.